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VIRTUAL STUDIO P O SITION SKNSING SYSTEM 

The present mvenaon relates to a virtual studio and in particular to 
a position sensing system for such a studio. 

5 

In virtual studio systems a chroma-key backgroimd is provided in 
front of which an actor stands. In the TV picture seen by the viewer the 
chroma-key background is replaced by a virtual scene. Such a system is 

1 described in PCT Patent Application No. WO95/30312 to the present 

10 applicant. 

In the above known system the position of the camera is calculable 
by using a coded pattern on the chroma-key background. This enables the 
position of the camera, as well as its orientation and tiie lens zoom, to be 
15 continuou^y calculated and thereby the p>erspective of the virtual 3D set 
can be adjusted correcdy to suit the camera position. 

This works perfectly for cameras which have a fixed focus and 
zoom or a fixed position but problems can be eiKX)imtered in situations 

20 where the cameraman h free to simultaneously change the camera position 
and the lens zoom. For example, when the camera's optical axis is 
perpendicular (or almost perpendicular) to the patterned wall, the changes 
in the captured pattern due to zooming of the lens or to changing the 
distance of the camera from the wall arc identical (or almost identical) and 

25 the system caimot extract the correct solution from the capmred pattern. 

The system of WO95/30312 also works well providing that the 
background panel is a patterned chroma-key panel. However the system 
does not work where a normal smdio or suge set with real background is 

30 used. 

I 



^irtiiai 3D object in a normal 
-n.,.. if It is desired to position a virtual 3U o j 
^ for a static camera because the 

3.ge set tins cannot be done except for ^^^^ ^ the 

posUion of the catnera relative to the stage set cannot be 

camera moves. 

U is a firs, objec. of a.c present ™» «- ^ 
a cn^era to be calculated m a non»al stage set u. wh 
comprise a real background. 

further object to euable vutua, 3D objects to be positioned 
in a normal sUgc set. 

represent mventionprcidesafi^er Object .0 enable ac«>rsw^ 

nd obiecu. to v.ear colours which would odterwuic dad. 
tcprescut foregroutxl object ^^^^^^^ ^ ^ 

. ..thanortnalc^on^^vJ^St.^- 

'trTir- daTTlue panems . sucb a prior 
::;rr:::e:sl..v,earb,uewblcb.se.e.ne.res.^^^^^^ 

background real set can be any colour 

The background real set does not have to be flat. 

,u accordance wlU, the present invenuoa a plural,,, of LED's are 
25 inaccoro , r ed's fonning a pattern which is known 

posttioned in a teal stage set. the LED fonn.^ I- 

an uutia. measu.ement of the pos.aon of each LED- 

The UED-s prefcabl, operate ,n the non-vtstble spccttun> to be 
,0 thereby uon detectable by a .«tnta, vtdeo camera. 



The LED outputs aie detected by a detector camera mounted on a 
TV camera, the pattern formed by the LHD's being utilised to compute the 
position of the TV camera. 

The present invenuon therefore provides a camera positioning 
system for a virtual studio, said system comprising a camera, an auxiliary 
detection device mounted on said camera, a plurality of light emitting 
devices (LED's) mounted in known positions to form a defined array and 
including electronic detection means connected to said detection device, 
said detection means comprising position sensing axialysis means for 
analysing the perspective of the defined array, as detected by said 
detection device, to provide a positional read out defining the position of 
said camera relative to the LED array. 

Preferably the camera is a television camera. 

Preferably the detector device comprises a detector camera which 
IS preferable boresighted to be aligned with the TV camera.. 

In a preferred embodiment the LED's are infra-red operating at a 
frequency outside the frequency range of the TV camera. 

Preferably each LED emits light in a coded form, thereby enabling 
each light source to be individually identifiable. 

Preferably tlie LED's are fixed to or positioned in a fixed 
relationship relative to a real scene. 

Preferably the LED's are all positioned in plane or a plurality of 

planes 
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emitting devices (LED's) mounted in known fixed positions to form a 
defined array within said scene and including electronic detection means 
connected to said detection device, said detection means comprising 
position sensing analysis means for analysing the perspective of the 
5 defined array, as detected by said detector device, to provide a positional 
read out defining the position of said television camera relative to the LED 
array and comprising a virmal object store, a perspective transformation 
unit and a combiner unit in which the positional read out fi-om the position 
sensing analysis means is connected to said perspective transformation unit 

10 to provide a transformation value, in which the virtual object store is 
connected to said combiner unit via said per^xxtive transformation unit 
to transform the perspective and size of a virtual object to be inserted into 
said combiner unit and in which the video output of the scene provided by 
said TV camera is connected to said combiner circuit and in which the 

15 combiner circuit provides as an ouqmt a combined video image in which 
the virtual object is positioned in said scene. 

The invention further provides a method for producing a combined 
video image comprising a real background scene viewed by a TV camera 
20 and a virtual reality foreground object, the method comprising the steps 
of: 

positioning a plurality of LED's in known fixed positions relative 
to the background scene, the LED's forming an array, 

detecting the positions of at least a proportion of the LED's relative 
25 to the TV camera, as the camera moves, at each position of the camera, 
computing the position of the camera relative to the background, 

selecting from a virtual reality store a virmal reality object to be 
positioned in the real background scene as viewed by the camera, 

selecting the position for the virtual reality object in the scene, 
30 transfomimg the size and perspective of the virtual reality object to 



c„„fonn.o<he calculate. pos,„onofa..can,e,areUtive,cU.ebac>cg.ouc^ 
v.n>. .e.,., o^ec. wi. a.e .sc^rou^ scene 
to form a composite picture. 

Bmbodtoen. of ..e prese.. mveat^n wUi oow ^ descriU. wia. 
reference lo a>e accompanying drawings in which : 

Frgure 1 shows schen,adcally a real baclcgronnd scene wi* ^»o 
^era. posiuoned to phou,graph or video the background scene; 

Figure 2 shows an altemadve schematic destgn of studio set m plan 

view; 

Figure 3 shows <hc stndio set of Figure 3 in elevat»>n; 
Figure 4 shows m block diagranunatic form circuitry suitable for 
tfie two camera system of Figure 1; 
,3 Figure 5 shows coding sequences suitable for the LED's; 

Figure 6 shows the scene of Figure 4 in plan view; and 
Figure 7 shows an alternative background scene in elevatron. 

With reference now to the drawings. Figure 1 shows a schematic 
.0 form of a smdio set 10 which compnses a backdoth .2 and possible stde 
clothes 14,16. These backdoths may compose, as in stage sets norma, 
pamted backCoths or could comprise acma. real objecu such as sta,xs. 

m the embodiment of Figure 1 the set ,s assume to comprise fUt 
back and side clothes but as ma, be seen. w,th referent to Figures 2 and 
3, real sets with solid objects may be used. 

With reference to Figure 1, the set ,s v,ewed by one or more 
cameras 20.22. Each first and second cameras 20.22 is provided w.th 
30 LED's A and B (200.220). 



First aad second static cameras 30,32 are provided preferably 
mounted above tlie set as shown They may be mounted on rails 300,320 
to be adjustable longitudinally and also in angle of view . 

5 The operation of LED's A and B and cameras 30,32 is to identify 

and track the position in \,y and 3 planes of the cameras 20 and 22. This 
operation is described in the co-pending British Patent Application No. 
9702636.3 to the present applicant which should be referred to with 
reference to the calculation of the position of cameras 20,30 relative to the 
10 set 10. The technical content of this prior application is incorporated 
herein by way of technical explanation. 

In this co-pending patent application, as well as in PCT No. 
WO95/30312. the "backcloih" comprises a chroma-key panel which is 
13 provided with a pattern enabling the perspective of the camera field of 
view relative to the panel to be continuously calculated. With respect to 
the calculation of perspective, reference is miadc to PCT No- 
WO95/30312. the technical content of which is incorporated herein by 
way of technical explanation. 

20 

Both the prior inventions have a practical problem because both 
require that the background panels are chroma-key. This prevents real 
sets being used and also prevents actors who represent foreground objects 
from wearing colours which arc the same as the chroma-key panels. 

25 

In the present invention, as shown in Figure 1, the set comprises 
a background panel which has real views/scenes painted thereon. These 
are exemplified by trees 120,122. 

30 In the present invention a plurality of further LED's 130 are 
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posiuoncd inthe panel. II m F as in FiEure 1 , 

L m.o surface of me panel. If panel eompr«es, F.gu • 
seimiuui ,A T FD's 130 are preferably 

a rear panel 12 and two s,de panels M,l6 ^ LED s 1 
positioned within such panels 

Each LED nray be identical hu, each LED is preferably switched 
. . coL s^L I thereby render >ts .dendty able to be read. Ea^ 
operates n. the mfra-red region to thereby no. be vrs.^le 
to the eye or to cameras 20,22. 

Eachcamera20..2ispreferab.ye,^PP«iwi,hannU^-redear^ 
^ r.2 which is bcresighted (i.e aligns with the o.in focal ax.s)-of the 
^20 or 22 The cameras 202,222 provkfc an inrage o, the axxay of 
eame,.20or22. 1 tagged depending on me distance 

lED'siaO.lhenuroberofLED slKingun^B 
,5 which me camera 20.22 is away from the panel 12.13.16. 

order to he able to correcdy analyse the position and perspective 

70 22 relative to the panel a plurality of LED's mnst to nnaged. 
::::erel.22.ant.c.pa.,oshowo.ypar.o,thep^.^^^^^ 
™ a larger number of LED. w.ll ^ retpnred. GcneraUy 7 LED -e 
coXed suffrcient and therefore a pane, may have between sev^ O) 
Tth-rty (30> or more LED. to ensure correct alignment calcnlauons. 

The system can also operate wrth a theatre se, such as shown in 
25 Figures 2 and 3- 

,„p,.res2and3.hese.comprisesage.^Uyc^^sUge5CO 
,„r outdoor settmg) wrth two p.Uars 502.504 and steps 506,508,510. 

The LED-s 130 are pos„.on«. m a flat plane bur the plane 



comprises portions of the steps 510 and pillars 502,504 as shown ij 
Figwe 3. 



With reference to Figure 2. the LED's associated with steps 510 are 
5 positioned slightly ui front of the steps as shown. This is to enable a flat 
plane of LED's to be established even if no actual piece of scenery exists 
at that exact position. 

It is also possible to position LED's on the side steps 506,508 
> providing the angle between the side steps and the main plane of LED's 
found by 502.504 and steps 510 is known. By angxUar mathematical 
analysis the position of the LED's 130 in both planes can be determined 
as can also be the case for LED's in steps 508. 

Thus, in the case of Figures 1 and 2.3 there can be positioned 
virtual objects such as table 132, chairs 134.136 (Figure 1) or a car 138 
as in Figure 3. It .s equally possible to position animated objects or 
displays in selected positions and for these to move naturally into new 
positions as cameras 20,22 move. 

With reference now to Figure 4. the inputs from cameras 
30,32.202.222 are fed to a computer unit 400 and to a tracking unit 402. 
Unit 400 can switch between cameras 20,22 to provide position 
information for either or both cameras. 

Unit 400 buffers the input from the cameras 30.32 to provide an 
output to tracking unit 402 which provides on line 4022 an output signal 
from the overhead cameras 30,32 which is fed to a position sensmg umt 
404. the output of which provide, on line 4042 x,y and z coordinates for 
each camera 20 or 22 Reference is here made to the above mentioned 



for a further explanation of 
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m combiner unit 418. 



The advantage of the system according to the present inveution is 
that a virtual object may be positioned within a real set because since the 
5 size and perspective of the virtual object and also its position relative to 
the set are known, the virtual object can be positioned within the set. If 
the object is m front of a part of the set then it will obscure the set and if 
behind it will be obscured by the set. 

10 Referring now to Figure 5, simplified digital codes are shown to 

illustrate a preferred embodiment. In this embodiment only four codes a), 
b), c) and d) are shown. Each LED is controlled by a control unit 424 to 
switch in accordance with a defined code. 

15 In Figure 5 four digital codes are shown 10000000, 10100000, 

10010000 and lOlOlOlOO. Obviously other combinations can be selected 
to provide for a plurality of LED's, all of which are identifiable by unit 
400. 

20 The position of each LED 130 in the real set must be known and 

this can be accomplished either by having camera 20 moved into a fixed 
known position and known focus and zoom and then calculating the 
position of each LED by reference to the read out of camera 202. 
Alternatively the LED's can be placed in known positions. 

25 

With reference to Figures 6 awl 7, the advantage of the present 
invention can be seen with a stage set 600 comprising a main backcloth 
602, first and second side (wing) cloths 604,606 and thiid and fourth side 
(wing) cloths 608. 610 

30 

11 



Each Clou. 602-610 n«y be pam«l as usual for a suge set. 

each com «.2-610 a nun^ber LED. be posiooned^ 
Those 00 00,^602 axe reference. BO «^ oaCol. «H,606 are .30 and 
cloths 608,610 are 130- . 

Each l^D «m have a coded CHitpu. in U» infra-red which wiU 
aererm^e no. on,, i. pos-.on . rhe elorh 602.«M.606,608 or 6,0bnr 
also which dod. .s loca.ed Thus, U« courpuu. progran. can de^rnnne 
O^e pos...on of d.e canrer, by reference .o USD's in core ^ one pUne. 

An advanuge here ,s d.. acrors can enter and ex., from dre wings 
via openings ind,ca.ed by arrows 612;614 in d.e nonna. manner and 
V j; obiecs can also be cade ro e«er and exit. Tl.ere,ore rhe sys.en. 
,s far nrore ver^ule d.a„ d,e prev,ous Icnown chronra-key systen,. 
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CLAIMS 

1. A camera positu^mng system for a virtual studio, said system 
comprising a camera, an auxiliary detection device mounted on said 
camera, a plurality of light emitting devices (LED's) mounted in known 

5 positions to form a defined anay and including electronic detection means 
connected to said detection device, said detection means comprising 
position sensmg analysis means for analysing die perspective of the 
defined array, as detected by said detection device, to provide a positional 
read out defining the position of said camera relative to the LED array. 

10 

2. A camera positioning system as claimed in claim 1 in which the 
camera is a television camera. 

3. A camera positioning system as claimed in claim 1 or claim 2 m 
15 which the detector device comprises a detector camera which is 

boresighted to be aligned with the camera. 

4 A camera positionuig system as claimed in any one of claims 1 to 

3 in which the LED's are infra-red operatmg at a frequency outside the 
20 frequency range of tlie camera. 

5. A camera positioning system as claimed in any one of claims 1 to 

4 in which each LED emits light in a coded form, thereby enabling each 
light source to be individually identifiable. 

25 

6. A camera positioning system as claimed m any one of claims 1 to 

5 in which die LED's are fixed to or positioned m a fixed relationship 
relative to a real scene 

30 7 A camera positioning system as claimed m claim 6 in which the 
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LED's are all positioned in plane or a plurality of planes 

8. A camera positioning system as claimed m claun 6 or claim 7 in 
which the real scene is a scene painted onto a background as used in 
theatres or is an actual location 

9. A camera positioning system as claimed in claim 8 in which the 
background scene may be any colour or combination of colours. 

10. A camera positioning system as claimed in any one of claims 1 to 
9 in which a plurality of television cameras are positioned in front of the 
background scene, each camera being provided with a detecting device. 

11. A method of determining the position of a camera relative to a 
background, the method comprising the steps of : 

positioning a plurality of LED's in known fixed positions relative 
to the background, the LED's forming an array; 

detecting the positions of at least a proportion of the LED's relative 
to Uie camera as the camera moves at each position of the camera; and 

computing the position of the camera relative to the background. 

12. A method of determinmg the position of a camera as clauned in 
claim 1 L the method Inrtlicr mcluding allocating to each LED in the array 
a specific code to thereby identify the LED and including the step of 

5 mterrogating the output of each LED position to identify the code. 

13. A method of determining the position of a camera as claimed in 
claim 12 in which each array comprises at least seven LED's positioned 
in known positions 



14. A method of determining the position of a camera as claimed in 
claim 12 in which each array comprises between 10 and 30 LED's 
positioned m known positions. 

15. A virtual reality system camera position system for a virtual 
studio, said system comprising a television camera providing a video 
output of a scene, an auxiliary detection device mounted on said television 
camera, a plurality of light emitting devices (LED's) mounted in known 
fixed positions to form a defined array within said scene and including 
electronic detection means connected to said detection device, said 
detection means comprising position sensing analysis means for analysing 
the perspective of the defined array, as detected by said detector device, 
to provide a positional read out defining the position of said television 
camera relative to the LED array and comprising a viitual object store, a 
per^jective transformation unit and a combiner unit in which the positional 
read out from the position sensing analysis means is connected to said 
perspective transformation unit to provide a transforxouation value, in which 
the virtual object store is coimccted to said combiner unit via said 
perspective transformation unit to transform the perspective and size of a 
virtual object to be insened into said combiner unit and in which die video 
output of the scene provided by said TV camera is coimected to said 
combiner circuit and in which the combiner circuit provides as an output 
a combined video image m which the virtual object is positioned in said 
scene 

16. A meUiod for producing a combined video image comprising a real 
background scene viewed by a TV camera and a virtual reality foreground 
object, the method comprising the steps of : 

posiiioiiing a plurality of LED's in known fixed positions relative 
to the background scene, the LED's forming an array, 

15 
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to form a composite picture 
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